The field of rendering three-dimensional (3D) objects or synthetic models can be partitioned into two main areas: polygon rendering and volume rendering. Polygon rendering is only concerned with the surface topology, whereas volume rendering also deals with the internal structure of objects or models.
Polygon Rendering
In processors and processes designed for polygon rendering, the surfaces of the object or model are described in terms of polygons, e.g., triangles and lines. These fundamental geometric components are used to generate constituent "fragments" representing the outer surface. A fragment is defined as all information (data) required to render ("draw") a single image pixel that is on the surface, for example, the x and y coordinates of the pixel, the red, green, and blue color values of the pixel, an alpha value representing opacity, depth values, texture coordinates, and the like. Fragments can be blended into a color (or image) buffer before being rendered as pixels.
Volume Rendering
In processors and processes designed for volume rendering, the basic building block is a 3D voxel. Depending on a viewing or projecting orientation (view frustum), voxels can be interpolated to yield volume samples. The interpolated volume samples can be considered to be equivalent to fragments as defined above. Volume rendering produces a base plane, or 2D array of color values, which can be considered equivalent to the image plane. This correspondence makes it easier to integrate polygon rendering with volume rendering and application programming interfaces of commonly used rendering systems.
One natural problem in 3D rendering, polygon or volume, is dealing with how best to represent the third dimension for purposes of comparing positions of objects, determining occlusion, clip, cropping, etc.
Depth in Polygon Rendering
For example, OpenGL, a commonly used graphic rendering tool, stores a single depth (z) value for each pixel located at a planar (x, y) position in the image buffer. There, each fragment is either blended into the image buffer or discarded, depending on a selectable comparison with the corresponding depth of the fragment, and the depth value associated with the corresponding pixel. Normally, the depth comparison operator is less-than, and a depth buffer is initialized with a predetermined background color and depth values. If the depth of the fragment is less than the corresponding depth, then the sample is blended, otherwise the sample is discarded.
In order to accommodate more complex applications, OpenGL allows the selection of any one of eight relational comparisons. For opaque fragments, the comparison operation causes the result to be the same as if the fragments were sorted before being rendered. For example, a less-than comparison selects fragments with smaller depth values and discards those with larger depth values. This is equivalent to sorting the fragments from back (larger depth) to front (smaller depth) with respect to the point of view.
With OpenGL, there is no good way to render translucent, depth-buffered triangles. Opaque triangles can be rendered in any order, and the blending under the control of the depth information provides a well-known method of removing hidden lines and surfaces from the scene. However for translucent triangles, the resulting value of a pixel in the image buffer depends on the order in which the triangles are presented. Therefore, in OpenGL, a common technique for rendering non-opaque geometries (polygons) is to first render opaque triangles, then to sort translucent triangles, and to last render the triangles in a back-to-front order. If two translucent triangles intersect in such a way that each partially occludes the other, then the triangles must first be partitioned into smaller non-occluding triangles before rendering.
In effect, the OpenGL depth buffer defines a dynamic clip surface. The depth buffer separates fragments that are nearer the image plane from those that are farther, making it possible to selectively discard fragments on this basis. Moreover, the depth buffer is dynamically updated during rendering, thereby "moving" the clip surface so that later fragments are handled differently from earlier fragments.
Another known method for rendering translucent polygons does not require the sorting and subdividing of the triangles. Instead, that method uses multiple rendering passes with two depth buffers. The two depth buffers are used to sort the translucent fragments. However, that method cannot be used with OpenGL, because OpenGL only supports a single depth buffer. Other methods sort triangles from front-to-back instead of back-to-front. A front-to-back order is more efficient because this order allows the equivalent of early ray termination, which involves ignoring objects that are occluded by nearer objects. All of these methods basically try to render a set of (initially) unordered triangles as if all of their fragments were rendered back-to-front or front-to-back at each pixel location.
Depth in Volume Rendering
While rendering a volume, it may be better to render front-to-back, rather than back-to-front. Front-to-back allows one to perform early ray termination. However, the difference between the two orders is simply a matter of efficiency: front-to-back order with Under blending should produce the same result as back-to-front order with Over blending. In many cases, the rendering order can be selected on a basis of personal preference. To be consistent with OpenGL, a depth buffer used during volume rendering should affect volume samples the same way that the OpenGL depth buffer affects fragments. The depth of each volume sample should be compared against the corresponding value in the depth buffer, and the color of the sample must then be either blended or discarded, depending on the result of the comparison. The initial values of the color buffer defines a background color or an image of some other object. The initial values must be blended behind the samples that are not discarded.
Complex Volume Rendering
There are many applications where some objects are represented as volumes and other objects are modelled as polygons. It is desired to depict all objects in the same final image. Moreover, these objects may interpenetrate each other. For example, markers, tools, and other objects modelled as polygons may need to be embedded into an object represented as a volume and must be rendered in correct juxtaposition with respect to the volume for any view direction. In an example medical application, a volume object representing a human heart, and another volume object representing the human lungs and bronchia must be rendered onto the same image so that the heart is in correct juxtaposition with respect to the lungs.
Consider the case where an opaque geometry is to be blended during volume rendering. In back-to-front terms, this can be done by first rendering the opaque geometry, then copying the resulting depth image to a depth buffer and base plane. Next, set the depth comparison to less-than, and render the volume from back-to-front. This sequence of operations discards volume samples that are behind the opaque geometry, in other words, volume samples that have depth values greater than or equal to the depth buffer. The Over operation is used to blend samples that are in front, i.e., samples that have depth values that are less than the depth buffer.
The same effect can be achieved with front-to-back volume rendering. Let Z.sub.G be a depth buffer, and let C.sub.G be a color buffer separate from the base plane. Proceed as follows: Render the opaque geometry using, for example, OpenGL, and copy the resulting image and depth values to C.sub.G and Z.sub.G, respectively. Next, initialize the base plane to RGB.alpha.=0. Follow this step by rendering the volume. For each sample, compare the z-coordinate of the sample's position to the corresponding value in Z.sub.G and discard the sample unless the sample passes the less-than test, unless the sample lies strictly in front of the depth buffer Z.sub.G. Blend the non-discarded samples into the base plane using the Under operation. As a final step, blend the contents of C.sub.G under the base plane, also using the Under operation. This produces the same effect as rendering back-to-front with Over and with the base plane initialized to the opaque geometry color.
These techniques are not restricted to opaque geometry, they can also support a single layer of translucent geometry as follows. Render the geometry in OpenGL. Copy the resulting RGB.alpha. and z-values to C.sub.G and Z.sub.G, respectively. Clear the base plane to RGB.alpha.=0. Render the volume as with opaque geometry, discarding all sample values that fail the less-than test against Z.sub.G. Continue until all rays terminate, either by failing the depth test, or by becoming opaque, or by reaching the end of the volume, whichever comes first.
Next, blend the contents of C.sub.G using the Under operation and store the result back into the base plane. Then in a second pass, do not clear the base plane. Render the volume with depth test greater-or-equal, i.e., discard samples that are in front of the depth buffer and accept those greater than or equal to it while blending with the Under operation. The result is a "sheet" of translucent geometry through the volume. Finally, blend the background color behind the resulting base plane. This can also be done in a single pass. Simply add another z-value test that means "blend geometry now," along with an additional set of blending stages to carry out the blending operation between two samples without losing any cycles.
However, these techniques have serious limitations. A single depth buffer does not support embedded multi-layer translucent (non-opaque) geometries. Some applications are likely to have opaque geometries, e.g., markers or prosthetics in medical applications. Other applications, such as geological surveys, can include embedded fault planes or other translucent geometry and markers. These translucent objects are likely to overlap, producing multiple layers of geometry to blend into the volume. Moreover, there are many possible mixes of opaque and translucent geometries that need to be handled while combining volume objects and polygon objects in the same scene.
Therefore, it is desired to provide a method and system for rendering volumetric objects and models including complex opaque and translucent internal structures.